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Abstract 

Experiments in coherent spectroscopy correspond to control of quantum mechanical ensembles guid- 
ing them from initial to final target states. The control inputs (pulse sequences) that accomplish 
these transformations should be designed to minimize the effects of relaxation and to optimize the 
sensitivity of the experiments. For example in nuclear magnetic resonance (NMR) spectroscopy, a 
question of fundamental importance is what is the maximum efficiency of coherence or polarization 
transfer between two spins in the presence of relaxation. Furthermore, what is the optimal pulse 
sequence which achieves this efficiency? In this letter, we initiate the study of a class of control 
systems, which leads to analytical answers to the above questions. Unexpected gains in sensitivity 
are reported for the most commonly used experiments in NMR spectroscopy. 
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1 Introduction 



The control of quantum ensembles has many applications, ranging from coherent spectroscopy to 
quantum information processing. In most applications involving control and manipulation of quan- 
tum phenomena, the system of interest is not isolated but interacts with its environment. This leads 
to the phenomenon of relaxation, which in practice results in signal loss and ultimately limits the 
range of applications. Manipulating quantum systems in a manner that minimizes relaxation losses 
poses a fundamental challenge of utmost practical importance. A premier example is the control of 
spin dynamics in nuclear magnetic resonance (NMR) spectroscopy JlJ. In structural biology, NMR 
spectroscopy plays an important role because it is the only technique that allows to determine the 
structure of biological macro molecules, such as proteins, in aqueous solution. In multidimensional 
NMR experiments, transfer of coherence between coupled nuclear spins is a crucial step. However 
with increasing size of molecules or molecular complexes, the rotational tumbling of the molecules 
becomes slower and leads to increased relaxation losses. When these relaxation rates become compa- 
rable to the spin-spin couplings, the efficiency of coherence transfer is considerably reduced, leading 
to poor sensitivity and significantly increased measurement times. 

With recent theoretical advances, it has become possible to determine upper bounds for the 
efficiency of arbitrary coherence transfer steps in the absence of relaxation [Q. However, from a 
spectroscopist's perspective, some of the most important practical (and theoretical) problems have 
so far been unsolved: 

(A) What is the theoretical upper limit for the coherence transfer efficiency in the presence of 
relaxation? 

(B) How can this theoretical limit be reached experimentally? 

The above raised questions can be addressed by methods of optimal control theory. The frame- 
work of optimal control theory was developed to solve problems like finding the best way to steer 
a rocket such that it reaches the moon e.g. in minimum time or with minimum fuel. Here we are 
interested in computing the optimal way to steer a quantum system from some initial state to a 
desired final state with minimum relaxation losses. In this letter we initiate the study of a class of 
control systems which gives analytical solutions to the above raised questions. It is shown that in 
contrast to common belief, widely used standard NMR techniques are far from being optimal and 
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surprising new transfer schemes emerge. 



2 Optimal control of nuclear spins under relaxation 

The various relaxation mechanisms in NMR spectroscopy have been well studied [Q, [3]. In liquid 
solutions, the most important relaxation mechanisms are due to dipole-dipole interaction (DD) and 
chemical shift anisotropy (CSA), as well as their interference effects (e.g. DD-CSA cross correlation 
terms) The optimal control methodology presented here is very general and can take into 
account arbitrary relaxation mechanisms. To demonstrate the ideas and basic principles we focus 
on an isolated pair of heteronuclear spins I (e.g. 1 H) and S (e.g. 13 C or 15 N) with a scalar coupling 
J. Both spins are assumed to be on resonance in a doubly rotating frame and only dipole-dipole 
relaxation is considered. This case approximates for example the situation for deuterated and 15 N- 
labeled proteins in H2O at moderately high magnetic fields (e.g. 10 Tesla), where 1 H- 15 N spin pairs 
are isolated and CSA relaxation is small. In particular, we focus on slowly tumbling molecules in so 
called spin diffusion limit [jj]. In this case longitudinal relaxation rates are negligible compared to 
transverse relaxation rates Q] . 

For such coupled two-spin systems, the quantum mechanical equation of motion (Liouville-von 
Neumann equation) for the density operator p is given by 

p = TTJ[-i2I z S z ,p]+Trk[2I z S z , [2I z S z ,p\]. (1) 

Here J is the scalar coupling constant and k is the transverse relaxation rate. This relaxation 
rate k depends on various physical parameters, such as the gyromagnetic ratios of the spins, the 
internuclear distance, and the correlation time of the molecular tumbling Q. In this letter, we 
address the problem of finding the maximum efficiency for the transfers 

I a 21^ (2) 

and 

I a — * Sp, (3) 

where a, f3, and 7 can be x, y or z. These transfers are of central importance for two-dimensional 
NMR spectroscopy and are conventionally accomplished by the INEPT (Insensitive Nuclei Enhanced 
by Polarization Transfer) |5| (see Fig. IA) and refocused INEPT || pulse sequence elements, re- 
spectively. 
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Figure 1: Transfer schemes for (A) INEPT and (B) ROPE (Relaxation Optimized Pulse Elements) 
for the transfer I a — > 21 pS^. Thick and thin arrows represent selective spin rotations by strong and 
weak rf pulses, respectively. Dashed arrows represent evolution under J couplings. 



The two heteronuclear spins have well separated resonance frequencies, allowing for fast selective 
manipulation of each spin on a time-scale determined by the coupling J and the relaxation rate k. 
Hence, in the following it is assumed that any initial Cartesian spin operator I a can be transformed 
to an operator of the form I x cos (3\ + I Z sin (3\ by the use of strong, spin-selective radio frequency (rf) 
pulses without relaxation losses (see Fig. 2). Let r±(t) represent the magnitude of polarization and 
in-phase coherence on spin / at any given time t, i.e. rf (t) = (I x ) 2 + (Iz) 2 , where (I a ) = trace{/? I a } 
represents the expectation value of I a . Using rf fields, we can exactly control the angle Pi in the 
term n (t) sin (3\ I z + r\ (t) cos (5\ I x . Hence we can think of cos /3i as a control parameter and denote 
it by Mi (see Fig. 2). 



</z> 




<2IyS Z > 



Figure 2: Representation of the system variables r\, ri-, the angles j3i, 02, and of the control 
parameters u\ — cos/3i, U2 — cos/?2 in terms of the expectation values (I x ), (Iz), (2IyS z ), and 
(2I Z S Z ). 



Observe that the operator I z is invariant under the evolution equation (1), whereas I x evolves under 
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the J coupling to 2I y S z and also relaxes with rate k. As the operator 2I y S z is produced, it also 
relaxes with rate k. By use of rf pulses it is possible to rotate the coherence operator 2I y S z to 
2I Z S Z , which is protected from relaxation (see Fig. 1 B). Let r-i represent the total magnitude of the 
expectation values of these bilinear operators, i.e. r\(t) = (2I y S z ) 2 + (2I Z S Z ) 2 . We can control the 
angle (3% in the term ^(t) cos ^2 2I y S z + 7-2 (i) sin/3 2 2I Z S Z and we define cos/?2 as a second control 
parameter ui (see Fig. 2). The evolution of r\(t) and r2(t) under the scalar coupling and relaxation 
can be expressed as [|| 

d_ 

dt 

Here 

£ = k/J (5) 

is the relative relaxation rate and measures the relative strength of the relaxation rate k to the 
spin-spin coupling J. 



r 2 (t) 



ttJ 



-U1U2 



ri(t) 
r 2 (t) 



(4) 



The central problem addressed in this paper is the following: Given the dynamical system in equation 
(4), how should Ui(t) and u 2 (t) be chosen so that starting from ri(0) = 1 we achieve the largest 
value for r 2 . In spectroscopic applications this would correspond to the maximum efficiency for the 
transfer of I a to 2IpS^ (Eq. 2). Observe if £ = (no relaxation), then by putting ui(t) — u 2 (t) = 1, 
we have ^2(57) = 1, i-e. after a time t — the operator I x is completely transferred to 2I y S z . 
This is the INEPT transfer element B. However if ^ > 0, it is not the best strategy to keep u\(t) 
and U2{t) both 1 (as demonstrated subsequently), see Fig. 3A. Using principles of optimal control 
theory, it is possible to obtain analytical expressions for the largest achievable value of r-i and the 
optimal values of u\{t) and U2(t), see solid curve in Fig. 3A. One of the main results of the paper 
is as follows: 

For the dynamical system in Eq. (4) the maximum achievable value of T2 (i.e. the maximum transfer 
efficiency 77) is given by 



v = Vi+e-z (6) 



and the optimal controls ul(t) and u%(t) satisfy the relation 

u* 2 (t) _ n(t) 
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Figure 3: (A) The dashed curve shows the trajectory of the dynamical system (4) when £ = 1 
and u\{t) and U2(t) are maintained at value 1 (INEPT transfer). The maximum transfer amplitude 
^inept - 1S reacne d a i i* — (4J)- 1 jjjj. The solid curve represents the trajectory for optimal choice 
of ui{t) and u 2 {t) (ROPE transfer). (B) Efficiency V INEPT of INEPT (dashed curve) and efficiency 
r\ of ROPE (solid curve) as a function of the relative relaxation rate £ for transfer (2). (C) Gain 
of ROPE transfer efficiency compared to INEPT-type experiments for transfer (2) (r]/r] INEPT ) and 
for the in-phase transfer (3) {rf n /rf efINEPT ). 



(The optimality of this choice of ui and u 2 can intuitively be seen by the fact that this maximizes 
the ratio of gain 5r 2 in r 2 to loss Sri in r\ for incremental time steps St. A more formal proof is given 
subsequently). The above result implies that throughout the optimal transfer process, the ratio of 
(2I y S z )(t) and (I x )(t) is always maintained at 

(2IyS Z )(t) 



= v, 



(8) 



(see Fig. 4). 



Experimentally, the relaxation optimized pulse element (ROPE) which achieves this optimal effi- 
ciency has the following characterization. Starting from the coherence operator I x (n = 1, r% = 0), 
this operator is immediately transformed to the polarization operator l z (which is protected against 
relaxation). Then the operator I z is gradually rotated towards I x (which relaxes and also evolves 
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to 2I y S z under the coupling term) such that Eq. (8) is fulfilled for all times. Once (I z ) becomes 0, 
the operator 2I y S z is gradually rotated to 2I Z S Z (which is also protected against relaxation), again 
maintaining the relation of Eq. (8) (see Fig. 4). Finally, 2I Z S Z is rapidly rotated to the target state 
27 (3 5 7 (see Fig. IB). 



A B 

</ z >. <2/ z 5 z > 




Figure 4: Schematic representation of the relation (8) to be satisfied by the optimal trajectory. (A) 
In the first period during which (I z ) > 0, the density operator p is restricted to the three-dimensional 
subspace spanned by the operators I x , 2I y S z and I z . (B) In the next period during which (I z ) = 0, 
the density operator p is restricted to the three-dimensional subspace spanned by the operators I x , 
2I y S z and 2I Z S Z . The optimal trajectory lies in the plane which satisfies Eq. (8). 



It is instructive to compare the optimum coherence transfer efficiency r\ (Eq. 6) for the ROPE 
transfer (solid curve in Fig. 3B) with the maximum transfer efficiency of INEPT which is fj INEPT = 
exp(-£ cot~ 1 (^))sin(cot~ 1 (^)) || (dashed curve in Fig. 3B). Figure 3C shows the ratio r]/r) INBPT 
as a function of £. In the limit £ 3> 1 the ratio r)/r] INEPT approaches e/2 = 1.359. 

For the transfer I x — > S x (g|) the operator I x is first transferred to 2I z S y which is then transformed 
to S x . The optimal transfer 2I z S y — > S x is analogous to the optimal transfer I x — > 2I z S y and 
has the same transfer efficiency. Therefore, the total efficiency for the in-phase to in-phase ROPE 
transfer is rj™ — r/ 2 . Fig. 3C also shows the ratio of this optimal efficiency versus the maximum 
efficiency of the refocused INEPT sequence fj re f- INEPT — (j] INEPT } 2 . In the limit of large £, the 
ratio approaches e 2 /4 = 1.847, i.e. gains of nearly 85 % are possible using relaxation optimized 
pulse elements (ROPE). 

The proof of the above results is based on the central tenet of optimal control theory, the principle 
of dynamic programming In this framework, to find the optimal way to steer system (4) from 
the starting point (ri,r 2 ) — (1,0) to the largest possible value r 2 , we need to find the best way to 
steer this system for all choices of the starting points (ri,r2)- Starting from (ri,r2), we denote the 
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maximum achievable value of r-i by V(ri,r2), also called the optimal return function for the point 
( r ij r 2)- The optimal return function for system (4) and optimal control Ux(ri,?*2) an d W2( r ii r 2) 
satisfy the well known Hamilton Jacobi Bellman equation, see || for details. It can be shown |u} 
that the optimal return function for the control system (4) is 



V(r 1 ,r 2 ) = \J rfrl + ■ 



(9) 



and the optimal controls satisfy the equation (Q). Evaluating the optimal return function at (1, 0), we 
get V(1,0) = 77. Therefore, the maximum transfer efficiency in a spectroscopy experiment involving 
transfer of polarization I x to 2I Z S X is rj and the optimal controls u\ and 112 satisfy Eq. (7). 

It is important to note that in the above problem, there is no constraint on the the time required 
to transfer I x to 2I Z S X . The maximum achievable efficiency obtained as a solution to the above 
problem can only be achieved in the limit of very long transfer times (although most of the efficiency 
in achieved in finite time). In practice, it is desirable to reduce the duration of the pulse sequence. 
Therefore this raises the question, what is the maximum transfer efficiency of I a to 2IpS~/ in a given 
finite time T. This problem can also be explicitly solved (see supplementary material). Here, we 
describe, the characteristics of the optimal pulse sequence: If T < cot - j 2 ^ then ui(t) — 1*2 (i) = 1 
throughout, i.e. fa and fa in Fig. 2 are always kept zero. This solution corresponds to the INEPT 
pulse sequence. For T > cot - j 2 ^ the optimal trajectory has three distinct phases (see Figs. 5 and 
6). 




Figure 5: Phase trajectory of the controls u\ and ui (panel A) and r(t) (panel B) for a finite-time 
ROPE sequence (f = 1). 
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Figure 6: Controls u\ and U2 (panel A), the corresponding rf pulse sequence (panel B) and and 
the expectation values (I z ), (Ix), (2IyS z ), and (2I Z S Z ) (panel C) are shown for a finite-time ROPE 
sequence (£ = 1, r = 0.1J -1 , T = 0.263J -1 ) that optimizes the transfer I x — > 2I y S z . In panel 
B, the initial hard 55^_ y pulse establishes ui(0) = 0.572 (see panel A) and the final hard 55°_ x 
pulse completes the transfer. During phase I and III, the optimal rf amplitudes B^/ y (t) are given 
in frequency units {v x ,y(t) = 1 1 B^ y (t) / 2tt , where 7/ is the gyromagnetic ratio of spin /). During 
phase II no rf pulses are applied. Approaching phase II (Panel B) the rf amplitude becomes large 
for a very short time period. This can experimentally be very well approximated by a hard pulse of 
small flip angle. 



There is a r (which is a function of T), such that for < t < r (phase I), u 2 (t) = 1 and Ux(t) 
is increased gradually from a value ui(0) < 1 to ui(t) = 1 (see supplementary material). Then for 
time t < t < T — t (phase II), the optimal control ui(t) — U2(t) — 1. Finally for t > T — r (phase 
III), we have u\(t) = 1 and U2 (t) is decreased from U\{T — t) = 1 to U2(T) = ui(0). The optimal 
control always satisfies u\{t) — U2(T — t), as depicted in Fig. 6A. The parameter r, is related to T, 
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through the following equation 



where 



cot 



T = 2t 



1 - «(t) 
2f«(r) : 



TTJ 



(10) 



tan 



1 - «(r) 
2£ ; 



k(t) = 1 + 2£ 2 - + £ 2 coth(7r J a/1 + $ 2 r + 2sinh- 1 £). 

At time r, the optimal trajectory {r\,r%) passes from phase I to II and makes an angle 9\ with the 
t*i axis and at time T — t the optimal trajectory passes from phase II to phase III and makes an 
angle # 2 with the ri axis (see Fig. 5B). The optimal efficiency r\T for the finite time T is expressed 
in terms of these angles as 

exp(^(0 1 -0 2 ))(l-esin2^ 2 ) 



sin(0i + 6» 2 ) 



(11) 



In the limit, T goes to infinity r = -j and #1 = 6*2 = tan -1 \/l + £ 2 — £ and r/x approaches 77 in 
(||). This corresponds to the unconstrained time case we discussed initially. For the general finite 
time problem, we can analytically characterize the optimal controls (see Fig. 6A) and the optimal 
rf pulse elements (see Fig. 6B) [jllJl . Fig. 7 depicts the maximum achievable efficiency as a function 
of transfer time T for various values of £. 




Figure 7: Optimal transfer efficiency tjt as a function of the total transfer time T for various values 
of £. The circles indicate the critical time ° ot v j 2 ^ below which the rope sequences are identical to 
the standard INEPT sequence. For times greater than this critical time, the ROPE sequences are 
more efficient than INEPT. 



3 Conclusions and Outlook 

In this letter, we initiated the study of a new class of control systems which arise naturally in op- 
timal control of quantum mechanical systems in the presence of relaxation. This made it possible 
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to derive for the first time upper achievable physical limits on the efficiency of coherence and polar- 
ization transfer on two coupled spins. In this letter, the focus was on the study of an isolated pair 
of scalar coupled heteronuclear spins under dipole-dipole relaxation in the spin diffusion limit. For 
this example a surprising new transfer scheme was found which yields substantial gains (of up to 
85%) in transfer efficiency. The results immediately generalize to the case of dipole-dipole and CSA 
relaxation in the absence of cross-correlation effects |L3]]. The methods presented here are by no 
means limited to the case of coupled two spins. These can be generalized for finding relaxation opti- 
mized pulse sequences in larger spin systems as commonly encountered in backbone and side chain 
assignments in protein NMR spectroscopy. Furthermore these methods directly extend to other rou- 
tinely used experiments like excitation of multiple quantum coherence JIJ . Some obvious extensions 
of the methodology presented here are to incorporate cross-correlation effects Q among different 
relaxation mechanisms and to include in the design of pulse sequences additional criteria such as 
broadbandedness and robustness with respect to relaxation rates and experimental imperfections. 
The methods presented here are not restricted to NMR applications but are broadly applicable to 
coherent control of quantum-mechanical phenomena in the presence of dissipation and decoherence. 
The control systems studied in this letter are characterized by the fact that they are linear in the 
state of the system and controls can be expressed as polynomial functions of fewer parameters. Such 
systems have so far not received much attention in the optimal control literature due to lack of 
physical motivation. It is expected that the study of these systems will foster further developments 
in the area of system science and mathematical control theory. 
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[8] From equation (@), we have d ( ^ >(t) = 0, d {1 *} {t) = -irJ {2I y S z ){t)-ivk (I x )(t), d {2I f t * ){t) = 
and d {2Iy d ^ Kt) = ttJ (4)(t) - Trfc (2I y s z )(t). Using the fact that r x (t) = \J {h) 2 {t) + {I x ) 2 {t) 
and r2(t) = y (2I y S z ) 2 (t) + (2I z S z ) 2 (t) and above set of equations, we can write 



d 
dt 



ttJ 



- j cos 2 01 (t) - COS 01 (t) COS 02 (i) 

cos /3i(t) cos 2 (t) - j cos 2 2 {t) 



nit) 



(12) 



[9] If we start at (ri,^), then by making a choice of controls in ^ and letting the dynamical 
system evolve, after small time St we can make a transition to all points (ri,r 2 ), which are 
related to (ri,r2), by 



h 






+ St IT J 






r 2 





-UiU 2 
U\U 2 — £u| 



ri 

' - 2 



From all points (fijfg) that can be reached by appropriate choice of (tii,ua) m small time 
(St, we should choose to go to that (fi,f 2 ) for which V(fi,f2) is the largest. But now note by 
definition of V that V(ri, r 2 ) — max ?i,r 2 V{fi, f 2 ). This can be re-written as 

V(ri,r 2 ) = maxU(ri + 8t{-^u\ri - uiu 2 r 2 ),r 2 + St(—^u 2 r 2 + Uiu 2 n)) 

for infinitesimal St. The right side of the above expression can be expanded (Taylor series 
expansion) in powers of St and retaining only the terms linear in St (for St approaching zero), 
we get 



V(n,r 2 ) = V(n,r 2 ) + feJmax [ ££ ] 
Let H = [ g£ 

L OTi 9r 2 J 



-£tt 2 -Ul«2 



'"1 
'2 



-£lt -TO 

TO — £v 2 



ri 

'2 



This equation then reduces to 
max H(ui , u 2 ) = 0. 



(13) 



The optimal control ui(ri, r 2 ) and 1*2(^1, f* 2 ) maximizes the above expression and its maximum 
value is zero. If we can find a function V(ri,r 2 ) , which satisfies equation dl3| ) then finding 
(ui,u 2 ) which satisfy ( p"3| ) will give us the optimal control to apply in any given state of the 
dynamical system. 



fK, A 2 = f£, a 4^ and/;: i^. Then 



[10] Let H(ui,u 2 ) be as in [9]. Let Ai 

H = — Airi[£ a6 u 2 + (6 — a)ui«2 + 

Observe if (a— b) < 0, then the only solution to equations ( |l3| ) is the trivial solution u* = u* 2 = 0. 
Therefore {a — b) > 0. Also note, when (a — fe) 2 < 4£ 2 afe, the only solution to equation ( |l3|) is 
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again the trivial solution. Therefore the only case for which d|) can be satisfied is if (b — a) 2 = 
4afr£ 2 , implying = ^/l + £ 2 — £. In this regime, maximizing H, we get = implying 
Integrating equation (^J) , for this choice of optimal control, we get that starting 



from the point {r\,r2), the optimal trajectory satisfies that ^(f) approaches ^vf-r 2 + r 2 for 
large t. This is then the desired optimal return function V r (r 1 ,r 2 ). It can be verified that the 
optimal return function satisfies equation (p"3|). 

[11] For < t < t, the optimal control is given by 

ui(t) = 



i? 2 {l + cosh(0(i))} 



(BRf + 2A 2 R 2 ) - R 2 cosh(0(i)) ' 

where A = sinh<^(^), B = cosh^(r) and 4>(t) — 2sinh _1 £ + 2wJty^l + £ 2 . The optimal trajec- 
tory crosses from region II to region III at the point 

(RuR,) = ( . 2 J =, . VT ( ) )■ 

^/tan 2 6» 2 + k(t) 

y tan 2 e 2 

(as depicted in Fig. 5). For t > t, we have iti(t) = 1 and U2(t) = u±(T — t). The explicit 
expression for v y for phase I in panel B of Fig. 6 in terms of U\ is 

Vy {t) = ttJ Ui {£(1 - Vl-u 2 ) + tanh(^)yiT^[l + ll^J }, 

and in phase III, v x (t) = v y (T — t). For the transfer I z — > 2I Z S Z , the flip angle of the initial 
and final hard pulses (see Fig. 6B) is given by sin -1 ui(0). For £ = 1 and T = 0,263 we find 
Ui(0) = .5716. The resulting value for initial and final flip angle is 55.138°. 

[12] In INEPT, the efficiency of the transfer I x — > 2I z S y as a function of transfer time t is given 
by T] INEPT (t) = exp(— irkt) sm(ir Jt). This efficiency is maximized for a transfer time t* = 
^jcot" 1 ^) and this value is r/ INEPT (t*) = exp(-£cot"H£)) sinM" 1 ^))- 

[13] In the presence of both CSA and dipole-dipole relaxation (with no cross-correlation effects) the 
evolution of the density operator takes the form 

p = nJ[-i2I z S z ,p]+Trk 1 [2I z S z , [2I z S z ,p}]+7rk 2 [I z , [I z , p}] + irk 3 [S Zl [S z ,p]\. 

In this case the operators I x and 2I y S z relax with effective rates k = k\ + k 2 and the operators 
2I z S y and S x relax with effective rates k' = k± + k%. Therefore the optimal efficiency of the 



transfer I a — » 2I z Sp is w 1 + (j) 2 — ~ and the optimal efficiency of the transfer 2I z Sp — > S a is 



/ i + (t) 2 -t- 
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4 Supplementary Material 



We rescale time to eliminate the factor it J in equation (^). Rewriting (|4|) in new time units we get 

,2 



d 


' n(t) - 




dt 







U\U2 ~C W 2 



n(t) 



(14) 



In the finite time case, the optimal return function V(ri,r2,t) has explicit dependence on time and 
by definition 

V(n,r2,t) — max V{r\ + 8t{—^u\ri — Mi^r^), r 2 + 5t(—£u 2 ,r2 + u\U2ri),t + St). 

Expanding again in powers of St , we obtain the well known Hamilton Jacobi Bellman equation 
dV 



dt 



max 



av ov 

dri dr2 



-U\U2 
U\U 2 — £lt2 



'2 



= 0. 



(15) 



As in |ic[ , let H = — Airi^u^ — (a — b)u\U2 + ^abu 2 ,]- Then equation ( |l5| ) can be rewritten as 



dV 

— — h maxH(ui,ji2) = 0. 
at u!,u 2 

For the finite time problem max Ul ^ 2 IHI > 0. This implies (a — b) 2 > 4£ 2 a6. We consider three 
separate cases for the problem 



1. Case I: If (a — b) < 2£, then the maximum of H is obtained for U2 — 1 and u\ = ^T- 

2. Case II: If (a — b) > 2£ and > 2£, then the maximum of H is obtained for u\ = 1 and 
u 2 = 1. 



3. Case III: If ^Ji < 2A then the maximum of H is obtained for u\ — 1 and = 

ab s ' 2£a 



-b 



From equation (p"5|), the adjoint variables (Ai, A2) = {jr~, S-) satisfy the equations Xi = and 



<9ri 



OH 
dr 2 : 





' Ai " 




" c 2 


— U\U2 




' Ai " 




. A 2 . 




uiu 2 






. A 2 . 



(16) 

where (Ai(T), A 2 (T)) = (0, 1). From equation we deduce that V = Ai7"i + A 2 r2 is a constant 

for optimal trajectory and equals the optimal cost r2{T) = Ai(0). Writing the equation for adjoint 
variables backward in time, let a = T — t then 



d 


' Ai " 






U\U2 




' Ai " 


da 






— UIU2 


-tu\ _ 
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where (Ai(cr), A2(o")) CT =o — (0,1)- Now u±(a) and 1(2(0") should be chosen to maximize \i{<j)\o=t- 



Observe this is exactly the same optimization problem as (14), where the roles of U\ and 112 have 



been switched. From the symmetry of these two optimization problems, we then have 

u{(t) = u*(T-t) 
n(t)=X 2 {T-t) ; r 2 (t) = Ai(T-t) 

«&(§) = 1 ; ^ = 2n(|)r 2 (|) 

Observe from (|lj, [l6|), that afr(t) is monotonically increasing and since ab(0) = and ab(-^) = 1, we 
have a&(t) < 1 for t < -j- Therefore u 2 (i) = 1 for t <%. Since 6(0) = 0, depending on a(0) we have 
two cases. Case A In this case > 1. Then we start in the case II discussed above and verify 
that in this case a — b is increasing for ab < 1. Therefore we stay in this case for all t € [0, -j] and 
therefore u\ = u 2 (t) — 1 for all t. Since b(0) = 0, we have b(-j) = tanT. Similarly, 

T _ a(0) + tan(f ) 
a( 2"^ ~ l-a(0)tan(^)' 

If ab(^) = 1 then above equation implies that tan(T) < 

Case B If < 1, then u*(0) = and the system begins in case I. Let n(t) satisfy 



k 2 — 2k + 1 



+ 2£k, k(0)=0. 



The solution to this equation is given by K (t) = 1 + 2£ 2 - 2^1 + i 2 coth(^l + i 2 t + 2/3), where 
sinh(/3) = £. It can be verified that in case I, the optimal trajectory satisfies -{€) = n(t). After time 
r, 3 ^r becomes equal to 1 and the system switches to case II. Putting = 1 and ~(t) = k(€), 
we get 7Y(7j = i1k(t) (denote this ratio by tan#i, see Fig 5, Panel B). Then again by symmetry at 
time T — r we have \ — \ ) = 1 and the system switches from case II to case III. In case III, verify 
~(t) = k(T — t) and the switching to this case occurs at tan #2 = 77 = • Thus the system 

spends T — 2t in region II. Then we have 

1 1 - k(t) , 2£k(t) 

T-2r = tan" 1 - tan" 1 S V ' 



2£ 1 - k(t) 

Thus providing result (|l0|). 

We now derive an explicit expression for r 2 (T). For t > T — r, 



= ^/r 2 2 (i) + «(r-t)r?(t), (17) 
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is constant along the system trajectories and equals the optimal return function r^iT). At t = T — r, 
we have ^(t-t) = tan 8 2 — 1 ~^ r ' > and therefore from ( [Tt| ) , we have 



V(T-t) = i?iA/sin 2 e»2 +cos 2 6» 2 - 2£ sin <9 2 cos 6> 2 , (18) 



where R 1 = y/r\{t) + r%(t) for t = T - r. Also note F(^) = 2n(~)r 2 (£ ). At time t = f , we then 
have 22- = tan( fll ^ 2 ) and therefore 

^(|)=i?2 s in(0 1+ 2 ) (19) 



where i? 2 = y r i (i") + r 2 (i")- Note between -j and T — r, the system evolves under ui = u 2 = 1. 
Therefore R\ — i? 2 exp(— — r)). Since V is constant, equating ( ^gh and (|l9|), we get equation 
0- 



1G 



